DOCUMENT RESUME 



ED 392 521 



PS 023 911 



AUTHOR 

TITLE 



INSTITUTION 
PUB DATE 
CONTRACT 
NOTE 



PUB TYPE 



Schmitz, Stephen 

PARADISU: Program Evaluation of the Micronesian 
Language Institute's Guam Family English Literacy 
Program. 

Guam Univ. , Mangilao. Micronesian Language Inst. 

Aug 95 
T003J20088 

33p.; Paper presented at the Annual Pacific Education 
Conference (12th , Koror , Repub 1 i c of Palau , Augus t 
1-3. 1995). 

Reports - Evaluative/Feasibility (1A2) — 

Speeches /Conference Papers (150) 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



MF01/PC02 Plus Postage. 

Academic Achievement ; ^Adul t Literacy; Elementary 
Educat i on; English Instruction; Family Programs ; 
^^Limited English Speaking; ^Literacy; ^Literacy 
Fducation; *0ut of School Youth: Parents; Program 
Descriptions; Program Effectiveness; Program 
Evaiuat ion; Social Services 

Chamorros; Chuukese People; ^Family English Literacy; 
^*^Guam; Palauan People 



ABSTRACT 

In 1992, the Micronesian Language Institute 
established the PARADISU program, whose purpose was to strengthen and 
deve lop the Engl ish literacy and school participation skills of, and 
the school survival and success of children from, Chamorro, Chuukese , 
and Palauan families living on Guam through direct instruction and 
other activities. The program served parents and out^of ^school youth. 
This report evaluates the program in its third and final year of 
operation to assess its product i vi ty , efficiency, and effectiveness . 
The evaluation was based on interviews of staff members and 
participants, observations of training sessions, review of documents, 
and data collected during formal evaluation. The program had four 
staff members Its activities included: (1) recruiting program 
parti c i pants from t radi t ional ly under served populat ions; (2) training 
participants in family literacy; (3) writing a literacy training 
manual; and (A) distributing a newsletter. The evaluator judged the 
program overall to be effective in its third year. Strengths of the 
program were the commitment and expertise of the staff and a training 
program focused on building the capacity of parents . Identified 
weaknesses included lack of coordinated effort within the program and 
with outside agencies, a chronic attendance problem, and unclear 
philosophy and strategy. These weaknesses, however, did not prevent 
effective completion of the program's goals of increasing the English 
proficiency of, and the understanding of the public school system by, 
Chamorro, Chuukese, and Palauan parents and out~of-school youth. The 
program was not renewed; nevertheless, recommendations for future 
literacy efforts are offered. These include considering the 
philosophy of the program; stressing cooperation, unity of purpose, 
and sharing of ideas; focusing greater resources on recruitment and 
assessment; and planning beyond the life of the program. (TM) 
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Executive Summary 

The Micronesian Language Institute's PARADISU, Title VII Family Literacy 
Project grant application was successful and was funded for a three-year cycle 
beginning July 1, 1992. The purpose of PARADISU is to sbengthen and develop the 
English literacy, school survival, school success, and school participation skills of a 
selected number of Chamorro, Chuukese, and Palauan families living on Guam 
through direct instruction and other program activities. 

This report presents evaluation research findings of the program in its third 
and final year of operation, based on observations of the program's operation 
throughout the year as well as data collected through the formal evaluative 
processes of May - July, 1995. The evaluator conducted on-site interviews of each 
s^dtf member and several parent participants and school officials. He also observed 
several training sessions and collected and reviewed a variety of PARADISU staff 
documents in order to build an assessment of the program's level of effort, 
productivity, efficiency, and effectiveness. 

The PARADISU program employed 4 regular staff members: 1 Project 
Director/ Literacy Trainer for Chamorro, 1 Palauan Speaker Literacy Trainer, 1 
Chuukese Speaking Literacy Trainer, and 1 Technical Assistant. PARADISU's 1994- 
95 program accomplishments included: 

• Carrying out many of the training activities that were planned for the proposal. 

• Convening a committed, knowledgeable and experienced staff. 

• Recruiting program participants from populations traditionally 
underrepresented in local family literacy efforts. 




•1 



2 



• Providing family literacy training to almost 600 participants. 

• Completing the Literacy Training Manual, including community background 
information, lesson ideas and resource materials used by each trainer, as well as a 
variety of resource materials translated into target-population languages. 

• Organizing and cataloging the collected resources of the PARADISU program. 

• Training a cadre of Literacy Aides to work in local Head Start programs. 

• Recruiting local classroom teachers for Family Literacy Night activities. 

• Writing and disseminating the MLI/PARADISU newsletter to enhance 
networking. 

Overall, the PARADISU program was judged to be effective in its third year of 
operation. The evaluator was able to identify various strengths about the program, 
including staffing, expertise, and parental involvement. Several weaknesses were 
identified, including lack of coordinated effort, a chronic attendance problem, and 
unclear philosophy and strategy. Ai’ihough these problems are worthy of note, they 
did not preclude the effective compl jtion of the overarching goals of the family 
literacy program. 

Several recommendations we-e made for future family literacy efforts. These 
recommendations included carefully considering the underlying philosophy of the 
program, working on cooperation, unity of purpose, and the sharing of ideas, 
focusing greater resources on recruitment and assessment, and planning beyond the 
life of the program. 
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I. PARADISU PROGRAM 



History of the PARADISU Program 

In the fall of 1991, in response to the local need and the call for new proposals 
from the U.S. Departoent of Education, the Micronesian Language Institute (MLI) 
of the University of Guan\ applied for federal grant assistance under the Title VII 
Family English Literacy Program authorized under the Bilingual Education Act (20 
U.S.C. 3281-3341). The MLI's grant application was successful and was approved for 
three years of funding through the Office of Bilingual Education and Minority 
Affairs (OBEMLA). PARADISU's funding cycle commenced on July 1, 1992. 

The purpose of PARADISU is to strengthen and develop the English literacy, 
school survival, school success, and school participation skills of a selected number 
of Chamorro, Chuukese, and Palauan families living on Guam through direct 
instruction and other program activities. 

The local needs and conditions in Guam that the program was conceived to 
address were described in the program's proposal document: 

• Language training opportunities which attempt to help Micronesian LEP 
adults with their need for basic survival English training on Guam are 
virtually nonexistent for Micronesians who have not graduated from high 
school (Woo, Aguilar, & Spencer, 1991). 

• Most LEP Chamorro, Palauan, and Chuukese adults living on Guam are 
not functionally literate in their first language. The reading a id writing of 
basic directions, letters, reports, basic international symbols and signs, 
maps, medical prescriptions, menus, invoices, and filling out of simple job 
applications, visa applications, immigration forms, etc. is a struggle in any 
language (Woo, Aguilar, & Spencer, 1991). 

• Their unfamiliarity with and inability to read and respond to school 
notices to the home, report cards, newsletters, invitations to school 
conferences, permission slips, school handbooks, progress reports of their 
children, etc, often get misinterpreted by school personnel. Judgments 
about the Micronesian parents and family by the educational 
establishment are often that "they" are simply "unreachable", "apathetic", 
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and/or "disinterested parents /fan\ily" (MRC 15 interviews with LOTE 
teachers, 1990, 1991). 

• Often due to poor con\n\unication, poorly developed relationships 
between the hon\e and the school, and son\etin\es culturally, out of 
respect for authority, Micronesian adults are often reluctant to forward 
con\n\ents to the school about their child's school progress, and to even 
ask basic questions about the local educational progran\. Logically, this 
situation is exacerbated by the fact that the vast n\ajority of Guan\ DOE 
instructional personnel are not proficient in any of the Micronesian 
languages ("Towards the Year 21000", MLI Board of Directors Forun\, 

1991). 

• In 1991, it was estin\ated that n\ore than 5,000 in\migrants fron\ other 
Micronesian islands now live on Guan\ (Guan\ Bureau of Planning, 1991.) 
Rubenstein & Levin (1992) suggested that n\ore than 7000 Micronesian 
n\igrants n\ay reside on Guan\, not including about 2,000 Palauans. 

• Approxin\ately 95% of all newly arriving Micronesian families are 
considered to be low-income (Guam Bureau of Planning, 1991). 

• The literacy rate among newly arriving Micronesian adults living on 
Guam is estimated to be less ^an 50% (Woo, Aguilar, & Spencer, 1991) 

• The incidence of alcoholism, unemployment, vehicular deaths associated 
with alcohol and drugs, suicide, and crimes of violence among 
Micronesian adults living on Guam are estimated to be disproportionately 
high compared with estimated for non-Micronesian adults (Guam Bureau 
of Planning, 1991) 

Description of the PARADISU PROTECT 

PARADISU in the Chamorro language is the word for "paradise". For the 
program, its name and logo symbolize the harmony which can be gained through 
learning, with literacy development as one of its vehicles. 



Staffing: The PARADISU program in FY 1994-1995 employed 4 regular staff 
members: 1 Project Director/Chamorro Literacy Trainer /Curriculum Developer, 1 
Palauan Speaking Literacy Trainer, 1 Chuukese Speaking Literacy Trainer, and 1 
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Administrative Assistant to help carry out all of its activities. (Attachment A of this 
report is a list of the employees and their respective roles on the project.) 



Major Objectives. Overall, the PARADISU program has eight (8) major objectives. 
They are: 



1.0 To provide direct instruction to Chamorro, Chuukese, and Palauan 
parents and out-of-school youth from non-literate or low literacy 
backgrounds with the intent of developing their proficiency in 
speaking, reading, and writing basic survival English. 

2.0 To provide direct instruction to Chamorro, Chuukese, and Palauan 
LEP parents and out-of-school youth with the intent of developing 
their understanding of basic policies, procedures, priorities, 
organization, concepts, activities, and expectations of the Guam public 
school system so as to be able to better advocate for their students' 
learning needs. 

3.0 To provide direct instruction to Chamorro, Chuukese, and Palauan 
LEP parents and out-of-school youth with the intent of developing 
their proficiency in understanding the language learning needs of their 
K-6 students to the degree that they are able to identify, organize, and 
successfully carry out a number of literacy promoting activities for K-6 
LEP students. 

4.0 To provide direct instruction and organization to Chamorro, 

Chuukese, and Palauan parents, out-of-school youth, and interested 
parents with the intent of developing them as Literacy Aides who 
could assume either paid or volunteer positions within the schools 
and various communities to facilitate: a) the development of 
proficiency in speaking, reading, and writing basic survival English 
among other LEP adults and out-of-school youth; b) the development 
of understanding of basic policies, procedures, priorities, organization, 
concepts, activities, and expectations of the Guam public school system, 
among others; and c) the development of proficiency in understanding 
the language learning needs of LEP K-6 students to the degree that th ;y 
are able to identify, organize and successfully carry out a number of 
literacy promoting activities for these students. 

5.0 To organize a network of committed organizations, agencies, and 
individuals concerned and involved with promoting literacy for 
parents and out-of-school youth throughout Guam to information- 
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share, joint problem-solve, promote public awareness, and strengthen 
current activities and efforts. 

6.0 To organize a network of committed organizations, agencies, and 
individuals all concerned and involved with promoting literacy for 
parents and out-of-school youth throughout Guam to make good use 
of the trained Literacy Aides who are an outcome of this program and 
who will be able to effectively work with their peers and others to 
strengthen their literacy skills and knowledge about school matters. 

7.0 To develop and refine a three-part curriculum for LEP parents and out- 
of-school youth which would address the following: a) their needs for 
a set of relevant activities, discussions, and practice trails aimed at 
developing their proficiency in speaking, reading, and writing basic 
survival English; b) their need for a sequence of lessons, discussions, 
activities, and sample materials for developing their understanding of 
basic policies, procedures, priorities, organization, concepts, activities, 
and expectations of the Guam public school system; and c) their need 
for training on a cluster of literacy promoting activities that they could 
successfully organize and carry out which should help develop the 
English language literacy skills of their students. 

8.0 To operate and manage this program as effectively and efficiently as 
possible so as to realize all the program's objectives; and to stress 
collaboration and cost-savings measures in the carrying out of all 
program activities so as to maximize the potential for improved 
literacy on Guam. 



PARADISU is organized around four (4) core interrelated strands under 
which all of the above program objectives are subsumed. They are: 



STRAND A: 
STRAND B: 
STRAND C: 
STRAND D: 



DIRECTION INSTRUCTION & LITERACY AID TRAINING 
FACILITATION OF NETWORK AND GUAM-WIDE LITERACY CAMPAIGNS 
CURRICULUM DEVELOPMENT 
PROGRAM MANAGEMENT 
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II. EVALUATION DESIGN 

This report presents evaluation research findings of the program in its third 
and final year of operation under the original OBEMLA grant. The evaluation is 
based on data collected in its fourth quarter of operation, 1995. The consultant 
selected by the PARADISU staff to conduct this evaluation has working familiarity 
with Guam and its various communities, and has an ongoing research agenda 
addressing education in Micronesia. 

In keeping with the tone and format of previous evaluations, the evaluator 
attempted to accomplish the following goals for the staff: 



• Provide another "set of eyes" for the staff from which to view their 
work. 

• Lend assistance in dociunenting the work of the PARADISU 
program. 

• Help provide feedback to the staff on current operations. 

• Assist the staff to gain greater alignment between what the written 
proposal states and what is actually occurring. 

• Note contextual problems that have occurred during the operation 
of the program. 

• Identify program strengths and weaknesses. 

• Help recommend various mechanisms, tools, and procedures 
which if adopted might help improve how the program is operated, 
perceived, managed, documented. 

• Summarize the accomplishments of the program for the third year 
and over the span of the three-year grant. 

• Provide suggestions and guidelines for future literacy efforts in the 
region. 
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The evaluator visited PARADISU program sites and activities throughout 
much of the program year, and attempted to assess the program based on long-term 
observations about the program as well as data collected during the formal 
evaluation period. The evaluator carried out a document review, several on-site 
observations of workshop sessions, individual interviews with each staff member, 
and review of the monitoring and quality control activities which the staff had 
conducted as part of their planned program activities. Some staff members had the 
opportunity to review the evaluation document before final submission; several 
comments and suggestions were incorporated into the final text. 

Evaluation Focus 

As in previous evaluations, an attempt to understand the effectiveness, 
efficiency, and appropriateness of the PARADISU program lead to a number of 
relevant questions. Answers to most of the questions were gathered from a number 
of sources. A summary of both the questions that formed the basic core of the 
evaluation and the method of inquiry used are listed in Attachment 6. 

Data Gathering Methods 

Formal evaluative data was gathered by the evaluator during May - July of 
1995. Each data gathering method employed for the evaluation progressed in the 
following manner: 

1. Semi-structured interviews with each staff member - Each PARADISU staff 
member was asked to respond to a similar set of questions. The same evaluator 
conducted all of the interviews. The list of questions contained approximately 10 
items. Questions included in the interview addressed: how well the project had 
completed their activities and met their objectives as delineated in their project 
proposals; their understanding regarding their role and the objectives of the 
PARADISU; problems each staff member had encountered in the fulfilling of 
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their perceived roles; how they viewed their abilities to serve tl.9 needs of the 
various community and parent groups; what program accomplishments they 
considered important or noteworthy; as an entire program, how well things had 
progressed organizationally, what had been accomplished, and what problems 
had been encountered. Each staff member vvas also invited to make whatever 
comments they wished to add to the evaluative data. 

Each appointment time was scheduled with the agreement of the individual 
being interviewed. Interviews ranged from 25 minutes to 80 minutes in length. 
While the interview was in session, the evaluator freely took notes. Upon each 
interview's completion, the evaluator further refined his notes. The evaluator 
met with individual staff members for a second time after the completion of the 
preliminary report to gather additional data and solicit additional comments. 
Som? staff were also interviewed by phone in order to clarify certain responses. 

2. On-site Observations of Training Events 

As in previous evaluations, the evaluator investigated the critical link between 
the program and the targeted parents and community members. The evaluator 
observed actual instruction to get a general sense of how the teaching and 
learning was taking place as a result of the program. The evaluatoi wanted to see 
how typical the instruction and use of instructional materials were within the 
program. The evaluator visited five training sessions at 3 schools, making every 
effort to not disrupt training that was in progress. The evaluator generally sat on 
the periphery of the main action to make observations while not disturbing the 
flow of the workshop. During two visits, the evaluator briefly interviewed three 
parents, four public school teachers, and an elementary school principal about 
their involvement in and perceptions of the PARADISU program. 
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3. Document Reviews - The Project Director made all the central program files 
available to the evaluators. The Administrative Assistant was available and 
helpful in locating various documents and materials. Among those documents 
that were reviewed were: the program proposal; the documentation of various 
activities by the project staff; previous evaluations of the program; and lists of 
the Guam community and network participants. A program manual for 
instruction was examined. Job descriptions and other materials from the initial 
planning phase of PARADISU were found to be out-of-date and inadequate in 
representing what was really going on. They were not used as a source of data. 

Overall Evaluation Procedures Taken for Evaluation: 

1. Met with PARADISU staff on individualized basis. 

2. Organized response data gathering around identified issues. 

3. Reviewed files and collected organizational data at PARADISU. 

4. Discussed "Organizational Health" and preliminary evaluation with staff 
and collected additional response data. 

5. Collected and summarized findings. 

6. Submitted final written report complete with recommendations to Project 
Director. 

7. Met with Project Director to clarify direction for follow-up. 
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III. EVALUATION FINDINGS 

OVERALL STRENGTHS OF THE PARADISU PROGRA M 

++ Commitment of individual PARADISU staff members 

Each staff member of the PARADISU program continued in the third year to 
exhibit a strong commitment to their respective communities and to the goals 
and methods of the PARADISU program. 

++ Cultural and pedagogical expertise of the PARADISU staff members 

The program was staffed by individuals who are knowledgeable about 
Micronesian cultures and able to demonstrate learning and teaching strategies 
appropriate to local needs. The staff was open, flexible, and willing to attempt 
innovative approaches to literacy. Because of the attitudes, cultural expertise, 
and professionalism of the PARADISU staff, PARADISU participants felt 
competent, important, and respected. 

+ + Training focused on building the capacity of parents 

The PARADISU Year Two Evaluation cited the overemphasis on passive 
learning and the presentation of information as a serious program weakness. 
This year, PARADISU staff realigned their strategies to emphasize the active 
participation of parents. Trainers worked to build a sense of efficacy among the 
community of parents. Participants became involved in training activities and 
were drawn into the literacy discussion. Parents worked on literacy activities 
side-by-side with their children. Parents were trained to take charge of family • 
literacy in the home and to become literacy aides in the Head Start program. The 
strategic move towards parental involvement and facilitative training was 
entirely successful and a strong addition to the program. 
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OVERALL WEAKNESSES OF THE PARADISU PRCXSRAM 

> Lack of collective effort in training activities and curriculum development 

The general feeling among staff during the third year was that staff had failed 
to resolve the chronic difficulty of working in isolation. Each staff member 
worked independently to carry out their specific program responsibilities, and 
staff carried out many training activities together. Each trainer brought their 
own specific training activities to training sessions, but there was little sharing of 
ideas and communal planning. Staff members seldom knew what each other 
planned to do before training sessions. This is not to imply staff friction—on the 
contrary, the warm atmosphere of caring and mutual respect was laudable. The 
cooperative spirit, however, failed to manifest itself in the sharing of ideas and 
expertise. 

There were many unfortunate consequences of this oversight. Collective 
feedback from peers would have built stronger activity sessions. Staff members 
each had a particular cultural expertise, but they failed to benefit from the 
aptitudes of others. Training session activities were not coordinated or 
organized in a thematic or sensical manner. 

> Failure to resolve a chronic attendance problem 

Each of the three years of the program, the evaluation has mentioned low 
attendance at training sessions. Reacting to attendance problems, the 
PARADISU staff changed emphases and strategies, yet inconsistent attendance 
continued to plague the program. The evaluator attended one session that had 
over 60 participants and another that had only two. 

Underlying the attendance problem was the strategy of relying on school 
principals and site personnel to promote the program and recruit participants. 

As the interest and commitment of principals and school personnel waxed and 
waned, so did program attendance. 

• Lack of a strategy to share expertise, materials, methods, and experiences with the 
larger professional community 
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To an outsider, it is startling how much knowledge is shared among the 
PARADISU staff. Over the three years of the program, staff have become 
impressively adept at understanding, relating to, and training program 
participants. PARADISU staff have amassed an impressive collection of 
strategies, materials, and resources pertinent to family literacy in Micronesia. As 
the program draws to a close, it is disappointing and disturbing to realize that the 
collective knowledge and collected resources of PARADISU are likely to die with 
the program. Staff have made no plans to distribute materials within the 
educational community or share their expertise with colleagues. 

Some sharing strategies were employed to involve the public schools in 
literacy activities that would continue beyond the program's life. Hopefully, 
parents and teachers will both see the value in continuing literacy training 
independent of the PARADISU program. The Micronesian Language Institute 
may be able to acquire funding to promote literacy and utilize the PARADISU 
collection in support of continued community literacy efforts. 

How Effective was the Program? 

PARADISU planned four integrated strands under which all of the program 
activities fell. Staff initiated program activities in Year One, then modified strategies 
in Year Two to enhance quality and concentrate services. During Year Three, staff 
fried to better coordinate the program, develop and document the program 
curricula, strengthen networking strategies, and assess program efficacy. The 
evaluation will address each of the four integrated strands in turn, highlighting 
significant accomplishments and noting problem areas. 

Direct Instruction 

Direct instruction was PARADISU's primary strategy for promoting literacy 
among the target populations. PARADISU staff were successful in designing literacy 
curricula and instructional programs that were appropriate for and effective with all 
target communities. The staff demonstrated a remarkable and laudable talent for 
crafting literacy instruction towards specific cultural expectations and the literacy 
needs of target groups. Classroom teachers became effective literacy educators in 
PARADISU-sponsored "Family Literacy Nights." A program manual was 
developed to document PARADISU's many materials and strategies and sponsor 
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program consistency and the sharing of ideas. The PARADISU materials collection 
was catalogued and organized. PARADISU successfully expanded literacy efforts to 
concentrate on the three groups targeted in the original project proposal. A Head 
Start program brought family literacy efforts to a new segment of the community. 

PARADISU trainers demonstrated a model of parent-community-school 
cooperative education that is new to Guam and the public school system. The 
program was unique because it introduced chronically underserved family 
populations to the schools that serve them, involving families and teachers in joint 
family literacy activities. By the end of the third year of the program, at least 1-2 
schools had begun to take independent responsibility for continuing family literacy 
activities beyond the life of the program. 

Problems derived from disruptions in the support of principals and school 
personnel and the consequent inconsistent attendance at PARADISU activities. 
PARADISU staff assessed client needs and program efficacy by polling parents but 
did not use the products of the assessment to significantly modify PARADISU 
program activities. 



Networking and Literacy Campaign Planning 

Many of the attempts at networking by PARADISU staff were successful. 
Although a formal literacy network was not organized, the Head Start effort resulted 
in the development of a group of literacy aides for the Guam schools. School 
teachers became involved in the "Family Literacy Nights" activities. Health officials 
were recruited to offer activities on family health to various literacy groups. Distinct 
Palauan and Chuukese literacy groups were established. The University of Guam 
became involved in the literacy effort through the College of Education, the College 
of Agriculture, and the University Extension Of:'ice. Networking became especially 
successful during the final year of the program when many earlier efforts came to 
fruition. 

During the third program year, efforts aimed at the Palauan and Chuukese 
communities were particularly successful. As ethnic minorities and immigrants to 
Guam, these groups were particularly resistant to PARADISU's early recruitment 
efforts. Staff emphasized group cohesion, developed a sense of community, and 
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capitalized on informal contacts to bring these groups into the family literacy 
network. 

Perceptions of the staff regarding networking and the literacy campaign, 
however, were ambivalent. Some staff felt great progress had been made in this 
area; others were frustrated by chronic attendance problems and the lack of support 
from Guam's Department of Education. Staff also voiced disappointment with the 
lack of progress in establishing a formal "literacy network." It is the opinion of the 
evaluator that much of the frustration about networking derived from ambiguous, 
unrealistic, and overly ambitious goals for the family literacy network. Clearly, each 
staff had different ideas about what "networking" entailed, yet these conceptual 
differences were not aired and explored. Furthermore, PARADISU staff never 
established a successful network among themselves (see Overall Weaknesses 
pg. 12). 

Curriculum Development 

PARADISU staff were highly successful in the area of curriculum 
development. During the first two years of the program, the staff amassed an superb 
collection of pedagogical strategies, literacy activities, and instructional materials 
related to Micronesian family literacy. Lessons, worksheets, and activities were 
compiled into a Literacy Training Manual; a bilingual Handbook of Literacy and 
Student Achievement Activities for the LEP Home was also finalized. Many 
activities were translated into the languages of target communities. During year 
thf^e, staff organized and catalogued developed materials while continuing new 
curricular development efforts. The professional expertise of the staff in the areas of 
curricular methods for LEP students, Micronesian cultures and family literacy was 
impressive. 



How Efficient was the Program? 

The program made good use of limited funds and manpower; there was little 
waste, duplication of effort, or overlapping programs. Collaboration and 
coordination of staff was evident when conducting workshops, but there was less 
effort expended on the sharing of ideas, instructional materials, and curriculum 
development. 




is 



16 



The organization of materials, files, resources, and the training manual were 
important steps in improving efficiency and sharing resources. Unfortunately, the 
newly-organized resources were underutilized. Many materials were bought at 
considerable cost. Others represented a great deal of development time. Yet, many 
materials were seldom, if ever, used. Public school teachers are unaware of 
PARADISU's vast collection of materials and the materials are not available for 
general distribution. 

Although the organization of activities was efficiently planned, attendance at 
training sessions was inconsistent. Many hours of careful planning were spent on 
Family Literacy Nights that were poorly attended, yet preparation was necessary 
because attendance was so unpredictable. Inconsistent attendance wasted valuable 
resources and discouraged and disheartened the staff. 

Several issues remained unclear to the evaluator regarding how the staff 
spent planning time. Staff seemed diligent and occupied themselves with 
preparation for training. Some staff complained of the time pressures of a heavy 
workload. Others were unwilling to take on new tasks like recruiting because they 
felt they could not spare time from planning. Yet, during literacy training, staff 
repeatedly presented the same activities. Seldom did staff spend more than 8 hours 
(a^ weekly actually involved in training. 

How Appropriate was the Program? 

A major strength of the PARADISU program was the alignment of program 
objectives with specific client populations. Staff had anticipated recruitment of a 
cadre of long-term participants, but modified training sessions to handle groups 
consisting primarily of newcomers. Staff frequently asked informal questions to 
assess participant needs in addition to relying on information gleaned from client 
needs-assessment surveys. The new Head Start literacy aide program was a 
particularly clever matching of program to population needs. 

Parents, students, pubic school teachers and principals spoke positively of the 
program. One principal mentioned the importance of children seeing their parents 
involved in school activities. A classroom teacher at a particularly well-attended 
session noted. 
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I can't believe all these parents are here. We had Parents' Day and almost 
nobody showed up. We really need parents to support us and work with 
their kids at home and it is really frustrating because they don't. This is great! 

A father attending his first Family Literacy Night explained, 

I came because my children asked me to. Now that I am here, I am glad I 
came. I guess school is different now--this looks kind of fun. 

What Obstacles Hampered Program Effectiveness? 

Many of the problems reported by staff grew out of the issue of weak 
coordination between PARADISU and other agencies, most notably the Department 
of Education. Networking difficulties, postponements, inconsistent attendance, 
locked buildings, planning difficulties, and several disappointments resulted from 
problems in identifying agency and school contacts and securing their cooperation. 
These circumstances were beyond the control of the PARADISU staff. 

PARADISU staff found themselves on the horns of a dilemma. PARADISU 
was charged with developing a self-sustaining capacity for literacy training among 
parents, teachers, and the Guam public schools system. PARADISU staff could not 
terminate their relations with outside agencies because it was these same agencies 
whose responsibility and capacity PARADISU was trying to develop. Yet, literacy 
efforts were often frustrated by the listless cooperation of these groups. In future 
efforts, staff should consider how to make PARADISU more independent of other 
agencies, especially in matters of client recruitment, while continuing to work 
within the interagency network. 

What were the Strengths and Weaknesses of the Various Strands? 

STRAND A: DIRECT INSTRUCTION 
Strengths and Accomplishments 

• Trainers organized, designed, and delivered training that addressed the 
educational needs, interests, and cultures of the participants. 

• The instruction targeted communities that have been underrepresented in past 
educational efforts. 
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• In the course of program year three, staff provided training and assistance to 
about 589 participants, an increase of more than 430 participants from year two. 

• Staff training of Head Start aides perpetuated literacy benefits beyond the life of 
the PARADISU program. 

• Other participants, most notably classroom teachers, were brought into the 
training process. 

• Family literacy training included children, parents, and public school teachers 
working together in literacy activities. 

• The program staff provided bilingual training and assistance to Palauan, 
Chuukese, and Chamorro families. 

Weaknesses and Identified Problems 

• Staff seldom shared ideas, strategies, or lessons. No time was set aside for staff to 
observe each other's training activities. Staff received little feedback from 
colleagues or other professionals. When the program was evaluated by other 
professionals, staff did not seem to use the feedback to significantly alter the 
strategies of the program. Staff neither observed the activities of the participating 
public school teachers nor made use of them in subsequent presentations. There 
seemed to be little difference between year two and year three activities. 

• Attendance at training sessions was inconsistent due to a reliance on other 
agencies for recruitment. PARADISU staff failed to use some obvious outreach 
approaches or suggest them to the schools. 

• Assessment and evaluation activities were not conducted on a regular basis or 
used as a basis for planning instruction. There were few opportunities for 
participant feedback. 

• There was no clear link between training activities and long term planning. 
Sessions appeared more as individual workshops than parts of a larger strategic 
effort. 
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STRAND B: NETWORKING AND LITERACY CAMPAIGN PLANNING 

Strengths and Accomplishments 

• Program staff continued to disseminate a newsletter and program brochure 
among members and community organizations. 

• Parents, teachers, and school administrators from Head Start and the Guam 
Department of Education were involved in training activities. 

• Network links with the University of Guam, community churches, the Guam 
Housing and Urban Renewal Authority (GHURA), and Head Start were 
continued and strengthened. 

• The program continued to solicit donations from community businesses. 

• Participation in the program by children, parents, and teachers increased 
dramatically. 

• The Parent Volunteei Program was successfully initiated; about 34 adult 
volunteers received training. 

Weaknesses and Identified Problems 



• Networking seemed to be an extemporaneous and informal activity; there was 
no apparent long-term networking strategy. 

• Formal networking links beyond the life of the PARADISU program were not 
established and solidified. PARADISU staff have not planned for what will 
happen after PARADISU is discontinued. Much of the program's emphasis on 
capacity building was overlooked while staff concentrated on literacy training 
activities. 
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• There is no formal mechanism to share PARADISU's ideas, materials, and 
resources beyond year three; accumulated knowledge and expertise may die with 
the program. 

• Classroom teachers contributed many interesting and creative activities to the 
training sessions. PARADISU staff, however, failed to note teacher-developed 
activities and subsequently integrate them into the PARADISU program. 



STRAND C: CURRICULUM DEVELOPMENT 

Strengths and Accomplishments 

• Staff completed a Literacy Training Manual containing background information 
on each of the three cultural communities (Palauan, Chuukese, and Chamorro), 
lesson plans, and resource materials. 

• Trainers created materials such as story booklets or literacy posters to assist in 
literacy training activities. Many of these activities were translated into 
languages of the target cultures. 

• Staff organized and catalogued the accumulated PARADISU literacy materials 
collection. 

Weaknesses j>nd Identified Problems 

• Many of the materials in the Literacy Training Manual and training activities 
were not adapted to emphasize target cultural groups and different learning 
styles. 

• There are no plans to share materials, resources, or the Literacy Training Manual 
with the larger teaching community. 

• There was little coordination and feedback between staff in the development of 
materials and resources. Each trainer had a group of personally-tailored 
activities for their own cultural group, but these ideas were seldom shared or 
modified for use with other groups. 
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• There was no overarching strategy or philosophy to the curriculum. Curriculum 
development, materials development, training and the Training Manual 
seemed to be more of a collection of individual activities than a coordinated plan 
of action. 

• Public school teachers were included as presenters in training sessions, but no 
effort was extended to document teacher participation or incorporate their 
strategies into later sessions. While many teachers learned from PARADISU, the 
staff seemed to learn little from teacher participants. 

STRAND D: PROGRAM MANAGEMENT 

Strengths and Accomplishments 

• Staff began to organize materials and document the work and progress of 
PARADISU's various efforts. 

• Basic program management systems continued to be reassessed and redefined. 

• Staff members documented each site's activities, progress, and attendance. 

• Staff finished the compilation of the Literacy Training Manual and ihe 
organization of materials and resources. Program files were centralized. 

Weaknesses and Identified Problems 

• Training reports were not completed in a standardized fashion. Training reports 
and activities were not shared among staff for the purposes of feedback, quality 
control or creative brainstorming. 

• Staff seldom reflected jointly about the goals, activities, or progress of 
PARADISU. 

• The monitoring of program i»ctivities was relatively absent in year three. Staff 
activities were inconsistently documented and seldom shared. 

• Planning and budgeting did not allow for the termination of the program. No 
decisions were made about what to do with program materials. There was no 
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follow-up of program activities. Some staff felt that their con . ibutions were not 
acknowledged. 



In summary, the development and presentation of family literacy training 
activities was both PARADISU's strength and its weakness. Training activities 
were effective and valuable for teachers and parents. Yet the focus on literacy 
training was often at the expense of other responsibilities such as recruitment, 
networking, and capacity building. 
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IV. CONCLUSIONS AND RECOMMENDATIONS 

PARADISU funding has been discontinued after the three years of the 
original grant. Many staff have made commitments to other programs and 
activities, and continuation of a modified PARADISU program awaits new grant 
funding. 

The following comments and recommendations are written for the benefit of 
staff members who may continue with a subsequent modified program as well as 
other others participating in future literacy efforts. Real progress was made over the 
life of the program and many lessons learned. The following conclusions and 
recommendations are made in the spirit of experimentation, sharing and 
cooperative learning manifest in the PARADISU program. 



STRAND A: DIRECT INSTRUCTION 

PARADISU was most effective in the area of direct instruction. Sessions were 
interesting, entertaining, and richly informative about literacy education. 
Participants left training sessions enthusiastic about the education of their children 
and competent in new literacy skills. Parents acquired a strong sense of efficacy as 
participants in and contributors to the education of their children. 

• PARADISU and the Micronesian Language Institute are based on the belief that 
literacy skills can empower individuals from disenfranchised cultures, and that 
bilingual education is a powerful vehicle for enhancing literacy. Staff assumed that 
everyone understood and agreed upon these key issues, but it is doubtful that 
everyone operated from the same set of assumptions. Staff began the program 
without serious reflection about philosophy, empowerment, literacy development, 
culture and community. Training activities and evaluations focused more on 
content than on processes of empowerment. Each trainer worked individually 
within their own community but there was no sense of strategic organization or 
unity of purpose. 

A guiding philosophy gives a program unity and focus. It provides an overall 
structure from which trainers can plan lessons that hang together strategically. 
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PARADISU staff themselves indicated that this overarching philosophy was lacking 
and did not provide a coordinated focus. 

• Those training sessions that were successful combined several key attributes: 

several staff offered a choice of individualized activities 
parents participated along with their children in the same activity 
classroom teachers were integrated into the training activities 
school administrators fully backed the program and helped recruit 
participants 

activities were process oriented and required active involvement 
the training session was highly organized and the pace was brisk 
there was no lecturing-staff and participants commurdcated as equals 
parents were made to appear competent in front of their children 

Unsuccessful training sessions suffered from several shortcomings: 

there were few choices of activities and many activities involved lecturing to 
parents about content areas 
few trainers were involved 

parents watched their children participate but did not become involved 
weak administrator support at the school caused disorganization and poor 
attendance 

classroom teachers appeared apatho*^ic or disinterested 
activities entailed passive participation 

parents were not at ease or appeared incompetent in front of their children 
sessions were disorganized; attending parents were given little direction 

• Program planning and assessment go hand in hand. PARADISU emphasized 
curriculum planning at the expense of thorough assessment. Many assessment • 
ideas v/ere fielded but few were followed through (e.g., individual portfolios). Staff 
were never c ^ely certain that the program effectively met client needs. 

STRAND B: NETWORKING 

• Attendance at training workshops was a problem over the life of the program. 
Planning, organization, instruction, and resource allocation were all impacted by 
inconsistent attendance. The staff strategy was to concentrate on training and 
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leaving recruibnent to site coordinators. This plan was ineffective because 
recruiting support was unreliable. Lack of recruitment resources put PARADISU at 
the mercy of other agents. Planning became passive and dependent. The most 
serious consequence of poor attendance was increasingly poor morale among 
trainers. 

• Networking begins at home. Although the idea of establishing a formal literacy 
network is laudable, it is difficult to translate networking objectives into a concrete 
plan of action. PARADISU staff made progress in enlisting the support of disparate 
client populations that are notorious for their very lack of participation in school 
programs. The capacity of the staff to network within the informal communities 
was laudable. The hesitancy of the staff to network at home— learning from each 
other and PARADISU's participating classroom teachers— was disappointing. 

• One goal of PARADISU was perpetuating the literacy network beyond the life of the 
program by building capacity among other agencies. Lack of planning beyond the 
life of the program frustrated the completion of this goal. PARADISU could have 
provided more impetus in this area. Several of PARADISU's initiatives, however, 
were also ignored or rebuffed by other agencies. 

• Planning and programming exclusively focused on what should transpire over the 
budget life of the program. Little attention was paid to several important questions 
about wrapping the program up. How might the community network be 
perpetuated? What should happen to program materials and the Literacy Training 
Manual! How might program expertise be transferred to other community 
agencies? 

• One aspect of literacy education is training parents and community resource 
personnel in the strategies and techniques of promoting family literacy. 

PARADISU's accomplishments in this area were notable. A closely related aspect of 
literacy is education and promotion— informing the community at large about the 
importance of literacy and the accessibility of literacy resources. PARADISU staff did 
not focus on this aspect of community literacy. 

Future program staff should consider carefully the feasibility of mounting a literacy 
campaign, utilizing local television, radio, parish bulletins, church groups, compact 
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impact agencies, public health services, and other service agencies to inform the 
larger community about the importance of literacy efforts. Radio and television 
programs, posters, letters, and other media could use Palauan, Chuukese, and 
Chamorro languages to build group cohesion and keep communities informed of 
literacy events in their native languages. Parents might have become more 
involved in literacy training activities had they understood the importance of 

. literacy in the home and their central role in family literacy. 

Admittedly, PARADISU staff hesitated to take the initiative in matters of client 
recruitment. After all, one of the goals of the program was to build this capacity 
within PARADISU's affiliates. Future efforts might resolve this dilemma by 
providing various agencies with materials, training, and support in the area of 
recruitment. A PARADISU sponsored inter-agency workshop on recruitment and 
networking might be an option. Another strategy might be joint PARADISU-agency 
recruitment activities. Such proactive methods involve and empower participating 
agencies. 

• One key to PARADISU's success was the ability of staff to keep a large program 
informal and personal. Many participants were visibly uncomfortable at the 
beginning of training sessions. Staff immediately put participants at ease by 
approaching them informally and communicating warmly and personally. 

Training was non-threatening. Participants did not appear at the center of attention. 
Participants were never put on the spot. 

• The Palauan and Chuukese trainers became particularly adept at exploiting group 
solidarity and group cohesiveness to make informal networking effective. Minority 
community status became an asset to networking and contributed to a sense of 
cultural dignity and pride. During training, participants were approached as cultural 
emissaries, cultural experts, and cultural preservationists, further enhancing their 
sense of worth and dignity. Linking literacy and empowerment to cultural pride 
and group solidarity was a notable breakthrough. 

• PARADISU initiated many evaluative strategies but assessment never became 
prescriptive in the sense of identifying community needs and devising strategies 
directly linked to meeting those needs. Futrre programs should investigate more 
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thoroughly the specific needs of target communities, and then devise strategies 
directly linked to those needs. 

/ 

In summary, the PARADISU program was evaluated to be successful. 

Especially in the third year of operation, substantial numbers of parents from 

specific target populations were successfully recruited into the literacy effort and 

trained in several literacy strategies via direct instruction. PARADISU successfully 

trained Head Start aides to perpetuate literacy training beyond the life of the 

program. Pedagogical coordination, assessment, and recruitment strategies were not 

entirely successful, and the PARADISU staff might have strengthened the 

p rogram's participation in each of these areas. These weaknesses, however, did not 

seriously detract from the overall efficacy of the program in providing literacy 
^ • 

traimng to the underserved Chamorro, Palauan, and Chuukese populations of 
Guam. 
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Attachment A 

PARADISU PROGRAM STAFF 

Program Year 1994-95 

ARLENE C S. DIAZ, 

PARADISU Project Director and Literaacy Trainer for Chamorro 

CANISIUS TKEL HLIBERT, 

Literacy Trainer for Palauan 

ZANDER DAVID REHLONG, 

Literacy Trainer for Chuukese 

ZINA CASTENEDA REYES 
Technical Assistant 
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Attachment B 



Core Questions of the Evaluation and 
Methods of Inquiry 



QUESTIONS: 



METHODS OF INQUIRY: 



MEASURES OF EFFECTIVENESS 



1. Did the program do what it said it 
would? Did the PARADISU 
complete all the activities it 
proposed ? 



2. Were the program objectives met ? 



3. Ability to gather and use feedback . 



1.1 Review program and program 
proposal against staff 
documentation and reports. 

1.2 Verify with community members 
and training participants. 

2.1 Look for what evidence exists that 
shows meeting of objectives. 

2.2 Discuss with staff what things they 
looked for as measures of success? 

3.1 Look at methods used to gather 
feedback. 



3.2 Review activities that were 

planned and/or modified based on 
feedback gathered. 
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MEASURES OF EFRCIENCY 

Review project management 
systems set in place. 

Review with staff their job 
descriptions. 

Review with each team member 
how well they understand the 
priorities and procedures of the 
total program. 

Review evidence of coordination 
among staff members. 

Review coordination between 
PARADISU and other Guam 
programs. 



MEASURES OF APPROPRIATENESS 



1. Did the program address the most 


1.1 


Review methods used to assess 


important needs? 




needs. 


2. Was the service delivery model 


1.2 


Review results of needs 


based on data gathered through real 




assessment. 


needs sensing activities? 


2.1 


Look at evidence that service 



delivery model was designed to 
address specific needs. 



1. Were the program and projects 1.1 

well managed? How many of the 
following were evident: well 
understood systems for 1.2 

information dissemination, 
problem solving, supervision, 
evaluation? Clarity regarding 1.3 

roles? 

2. Was the program cost-effective? 

Were duplications of effort kept to 

a minimum? Where appropriate 2.1 

was there collaboration and 
coordination among the individual 
projects? 2.2 



O 
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